An Ensemble method for Content Selection for Data-to-text Systems

We present a novel approach for automatic report generation from time-series data, in the context of student feedback generation. Our proposed methodology treats content selection as a multi-label classification (MLC) problem, which takes as input time-series data (students' learning data) and outputs a summary of these data (feedback). Unlike previous work, this method considers all data simultaneously using ensembles of classifiers, and therefore, it achieves higher accuracy and F- score compared to meaningful baselines.Comment: 3 pages, 2 figures, 1st International Workshop on Data-to-text Generatio

A Review of Evaluation Techniques for Social Dialogue Systems

In contrast with goal-oriented dialogue, social dialogue has no clear measure of task success. Consequently, evaluation of these systems is notoriously hard. In this paper, we review current evaluation methods, focusing on automatic metrics. We conclude that turn-based metrics often ignore the context and do not account for the fact that several replies are valid, while end-of-dialogue rewards are mainly hand-crafted. Both lack grounding in human perceptions.Comment: 2 page

The interaction between voice and appearance in the embodiment of a robot tutor

Robot embodiment is, by its very nature, holistic and understanding how various aspects contribute to the user perception of the robot is non-trivial. A study is presented here that investigates whether there is an interaction effect between voice and other aspects of embodiment, such as movement and appearance, in a pedagogical setting. An on-line study was distributed to children aged 11–17 that uses a modified Godspeed questionnaire. We show an interaction effect between the robot embodiment and voice in terms of perceived lifelikeness of the robot. Politeness is a key strategy used in learning and teaching, and here an effect is also observed for perceived politeness. Interestingly, participants’ overall preference was for embodiment combinations that are deemed polite and more like a teacher, but are not necessarily the most lifelike. From these findings, we are able to inform the design of robotic tutors going forward

A Reusable Interaction Management Module: Use case for Empathic Robotic Tutoring

We demonstrate the workings of a stochastic Interaction Management and showcase this working as part of a learning environment that includes a robotic tutor who interacts with students, helping them through a pedagogical task

WoZ Pilot Experiment for Empathic Robotic Tutors: Opportunities and Challenges

We discuss the challenges and opportunities in building empathic robotic tutors based on a preliminary Wizard-of-Oz (WoZ) pilot study. From the data collected in this study, we identify situations where empathy in a robotic tutor could have helped the conversation between the learner and the tutor. The video presented with this paper captures these situations where two children participants are interacting with a map application and a robot tutor operated by a wizard

How Expressiveness of a Robotic Tutor is Perceived by Children in a Learning Environment

We present a study investigating the expressiveness of two different types of robots in a tutoring task. The robots used were i) the EMYS robot, with facial expression capabilities, and ii) the NAO robot, without facial expressions but able to perform expressive gestures. Preliminary results show that the NAO robot was perceived to be more friendly, pleasant and empathic than the EMYS robot as a tutor in a learning environment

MIRIAM: A Multimodal Chat-Based Interface for Autonomous Systems

We present MIRIAM (Multimodal Intelligent inteRactIon for Autonomous systeMs), a multimodal interface to support situation awareness of autonomous vehicles through chat-based interaction. The user is able to chat about the vehicle's plan, objectives, previous activities and mission progress. The system is mixed initiative in that it pro-actively sends messages about key events, such as fault warnings. We will demonstrate MIRIAM using SeeByte's SeeTrack command and control interface and Neptune autonomy simulator.Comment: 2 pages, ICMI'17, 19th ACM International Conference on Multimodal Interaction, November 13-17 2017, Glasgow, U

A Study of Automatic Metrics for the Evaluation of Natural Language Explanations

As transparency becomes key for robotics and AI, it will be necessary to evaluate the methods through which transparency is provided, including automatically generated natural language (NL) explanations. Here, we explore parallels between the generation of such explanations and the much-studied field of evaluation of Natural Language Generation (NLG). Specifically, we investigate which of the NLG evaluation measures map well to explanations. We present the ExBAN corpus: a crowd-sourced corpus of NL explanations for Bayesian Networks. We run correlations comparing human subjective ratings with NLG automatic measures. We find that embedding-based automatic NLG evaluation methods, such as BERTScore and BLEURT, have a higher correlation with human ratings, compared to word-overlap metrics, such as BLEU and ROUGE. This work has implications for Explainable AI and transparent robotic and autonomous systems.Comment: Accepted at EACL 202